Preserving authentication under item change

ABSTRACT

Apparatuses and methods associated with preserving authentication under item change are disclosed herein. In embodiments, acquiring digital image data of an image of at least a portion of a target physical object; extracting features from the image data to form a digital fingerprint; querying the database system to seek a matching record based on the digital fingerprint; based on an amount of difference between the digital fingerprint and a stored digital fingerprint of the database, update the database system to output a new indication of a new match to the physical object for any new samples that are not matchable to the stored digital fingerprint within a first predetermined similarity threshold provided the new samples are matchable to the digital fingerprint within a second predetermined similarity threshold. Other embodiments may be disclosed or claimed.

RELATED APPLICATIONS

This application is a non-provisional of, and claims priority pursuantto 35 USC § 119(e) to the following applications: U.S. provisionalapplication No. 62/297,262 filed Feb. 19, 2016, U.S. provisionalapplication No. 62/297,285 filed Feb. 19, 2016, U.S. provisionalapplication No. 62/297,334 filed Feb. 19, 2016, and U.S. provisionalapplication No. 62/297,343 filed Feb. 19, 2016. All of theaforementioned U.S. provisional applications are hereby incorporated byreference as though fully set forth.

U.S. Non-provisional patent application Ser. No. 14/531,307, entitledDIGITAL FINGERPRINTING TRACK AND TRACE SYSTEM, filed on Nov. 3, 2014,now issued as U.S. Pat. No. 9,582,714, is incorporated by reference asthough fully set forth.

Copyright © 2016-2017 Alitheon, Inc. A portion of the disclosure of thispatent document contains material which is subject to copyrightprotection. The copyright owner has no objection to the facsimilereproduction by anyone of the patent document or the patent disclosure,as it appears in the Patent and Trademark Office patent file or records,but otherwise reserves all copyright rights whatsoever. 37 CFR §1.71(d).

BACKGROUND

Counterfeiting of manufactured goods is a worldwide problem, with recentstudies estimating that 8% of the world's total GDP is now generated bythe manufacturing and sales of counterfeit products. Many classes ofcounterfeit goods create substantial risks to public health includingcounterfeit pharmaceutical drugs, auto parts, pesticides, and children'stoys. In addition, counterfeit computer chips, aerospace parts, andidentification documents present significant risks to national security.

Authentication alone is not enough to stop counterfeiting.Counterfeiters use a variety of strategies, including divertingunfinished products from factories that make authentic goods and thenadding their own counterfeit brand identifiers such as labels and tags.Counterfeit items can enter the supply chain at any point, including atthe original manufacturing facility, at the shipper, in distribution, orin retail stores. Unless the manufacturer or supplier can identifyexactly where and when the item entered the supply chain, identifyingand eliminating the counterfeit goods can be almost impossible.

Many different approaches have been tried to uniquely identify andauthenticate objects, including labeling and tagging strategies usingserial numbers, bar codes, holographic labels, RFID tags, and hiddenpatterns using security inks or special fibers. All of these methods canbe duplicated, and many add a substantial extra cost to the productionof the goods sought to be protected. Physical labels and tags can alsobe easily lost, modified, or stolen.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above-recited and otheradvantages and features of the disclosure can be obtained, a moreparticular description follows by reference to the specific embodimentsthereof which are illustrated in the appended drawings. Understandingthat these drawings depict only typical embodiments of the invention andare not therefore to be considered to be limiting of its scope, theinvention will be described and explained with additional specificityand detail through the use of the accompanying drawings in which:

FIG. 1A is a simplified flow diagram illustrating a method for creatinga storing a digital fingerprint of an object in a database.

FIG. 1B illustrates a process that includes more robust featureextraction.

FIG. 2 is a simplified flow diagram illustrating a method for matching adigital fingerprint of a target object to a database of existing digitalfingerprints.

FIG. 3 is a simplified conceptual diagram showing scanning of an objectat various times and places along a manufacture and distribution chain.

FIG. 4 is a simplified conceptual diagram illustrating a database systemand use of a mobile device application to query authenticationinformation related to an object.

FIG. 5 is a simplified flow diagram illustrating a method for trackingan object to verify its provenance.

FIG. 6 illustrates an example of authentication region and objectfeature definition for a U.S. passport.

FIG. 7 illustrates a process of re-authenticating an object based onauthenticating an aggregate.

FIG. 8 illustrates a process of tracking objects and aggregates thereof.

FIG. 9 illustrates a process of creating a personal history forcomponents of a system, such as chips of a processing card.

FIG. 10 illustrates a process of creating a personal history for objectsof a supply chain aggregation, such as wine bottles of a wine caseand/or shipping container.

FIG. 11 illustrates a process of preserving a level of confidence ofauthenticity of a physical object.

FIG. 12 illustrates a process of preserving a level of confidence ofauthenticity of a physical object on induction into a tracking system.

FIG. 13 illustrates a process of preserving authentication under itemchange.

FIG. 14 illustrates a process of classifying item change.

DETAILED DESCRIPTION

In this application, we use the term “scan” in a broad sense. We referto any means for capturing an image or set of images, which may be indigital form or transformed into digital form. The images may be twodimensional, three dimensional, or be in the form of a video. Thus a“scan” may refer to an image (or digital data that defines an image)captured by a scanner, a camera, a specially-adapted sensor array suchas CCD array, a microscope, a smart phone camera, a video camera, anx-ray machine, etc. Broadly, any device that can sense and captureelectromagnetic radiation that has traveled through an object, orreflected off of an object, is a candidate to create a “scan” of theobject. Various means to extract “fingerprints” or features from anobject may be used; for example, through sound, physical structure,chemical composition, or many others. The remainder of this applicationwill use terms like “image” but when doing so, the broader uses of thistechnology should be implied. In other words, alternative means toextract “fingerprints” or features from an object should be consideredequivalents within the scope of this disclosure.

Authentication Regions

In an embodiment, individual objects are scanned and a unique digitalsignature is generated by a digital fingerprinting method that utilizesthe object's natural structure or features. The object is registered inthe system database. Once the object is registered, the DigitalFingerprinting Track and Trace System can track the location of anyobject as it passes through a supply chain, distribution network, orsales channel. The system maintains a database record for each uniqueobject, and can store any relevant data related to the object over thecourse of its lifetime. The system can be queried, generate reports, andanalyze data on individual objects or on sets of objects. Applicationsof the system include but are not limited to object authentication,determining the provenance of an object, creating audit trails, andidentifying where counterfeit goods are entering manufacturing,distribution or sales networks.

In other applications, an original digital fingerprint can be comparedto a subsequent digital fingerprint of an object to establish that theobject is the original, without regard to its history of locations orpossession since the original digital fingerprint was acquired andstored.

Another aspect of some embodiments relates to detecting a counterfeit orforged object, for example a document such as a drivers license orpassport. In this case, there may be no “original” or source objectdigital fingerprint for comparison. Rather, “fingerprints” of knownindicia of counterfeit or forged objects can be acquired and stored. Forexample, a large number of bogus New York State driver's licenses mightbe obtained by law enforcement officials in a raid or the like. Digitalimages of those forged documents can be acquired, and analyzed to formdigital fingerprints, as described in more detail below.

In an embodiment, “Forgery feature vectors” can be collected and storedin a database, for example, sharp, non-bleeding edges where a photographhas been replaced or torn paper fibers where an erasure occurred. Thesefingerprints can be searched and compared to detect a forged document. Acount of “fraud indicator matches” can be compared to an empiricalthreshold to determine a confidence that a document is forged (or not).

Because digital fingerprinting works with many different types ofobjects, it is necessary to define what parts of the digital images ofthe objects are to be used for the extraction of features forauthentication purposes. This can vary widely for different classes ofobjects. In some cases it is the image of the entire object; in othercases it will be a specific sub-region of the image of the object. Forinstance, for a photograph we may want to use the digital image of theentire photograph for feature extraction. Each photograph is different,and there may be unique feature information anywhere in the photograph.So in this case, the authentication region will be the entirephotograph.

Multiple regions may be used for fingerprints for several reasons, twoof which are particularly important. It may be that there are severalregions where significant variations take place among different similarobjects that need to be distinguished while, in the same objects, theremay be regions of little significance. In that case a template may beused (see below) primarily to eliminate regions of little interest.

A bank note, for example, can be authenticated if a few small arbitraryregions scattered across the surface are fingerprinted, along withrecognizing the contents of a region telling the value of the bank noteand one containing the bank note's serial number. In such a case thefingerprints of any region (along with sufficient additional informationto determine the bank note's value and its purported identity) may besufficient to establish the authenticity of the bill and multiplefingerprinted regions are used solely in the event that one or moreregions may be absent (through, for example, tearing) when the bill islater presented for authentication. Sometimes, however, all regions ofan item must be authenticated to ensure the item is both authentic andhas not been altered.

A passport provides an example of feature extraction from anauthentication region; see FIG. 6. On a passport, the features that wemay want to use for authentication may be extracted from regionscontaining such specific identification information as the passportnumber, recipient name, and recipient photo, as illustrated in FIG. 6.In that case one may define a feature template specifying those regionswhose alteration from the original would invalidate the passport, suchregions including the passport holder's photo and unique personal data.

The ability to define and store the optimal authentication region for agiven class of objects offers significant benefits to the user, althoughit is not mandatory. In many cases it is much easier to scan a limitedregion of an object than the entire object. For instance, in the case ofan article of designer clothing, it is much easier to take a picture ofthe manufacturer's label than it is to take a picture of the entiregarment. Further, defining such regions enable the detection of partialalteration of the object.

Once an authentication region is defined, specific applications can becreated for different markets and classes of objects that can assist theuser in locating and scanning the optimal authentication region. Forinstance, an appropriately sized location box and crosshairs canautomatically appear in the viewfinder of a smartphone cameraapplication to help the user center the camera on the authenticationregion, and automatically lock onto the region and take the picture whenthe camera is focused on the correct area. It should be noted that whilesome examples discussed above are essentially two-dimensional objects(passport, bank note), the present disclosure is fully applicable tothree-dimensional objects as well. Scanning or image capture may be 2-D,3-D, stereoscopic, HD etc. Image capture is not limited to the use ofvisible light and fingerprint data capture is not limited to images.

In many cases, objects may have permanent labels or other identifyinginformation attached to them. These can also be used as features fordigital fingerprinting. For instance, wine may be put into a glassbottle and a label affixed to the bottle. Since it is possible for alabel to be removed and reused, simply using the label itself as theauthentication region is often not sufficient. In this case we maydefine the authentication region to include both the label and thesubstrate it is attached to—in this case some portion of the glassbottle. This “label and substrate” approach may be useful in definingauthentication regions for many types of objects, such as consumer goodsand pharmaceutical packaging. If a label has been moved from it'soriginal position, this can be an indication of tampering orcounterfeiting. If the object has “tamper-proof” packaging, such areasas may be damaged in attempts to counterfeit the contents may also beuseful to include in the authentication region.

In some cases, we will want to use multiple authentication regions toextract unique features. For a firearm, for example, we might extractfeatures from two different parts of the weapon. It is, of course,important that both match the original but since the two parts may bothhave been taken from the original weapon and affixed to a weapon ofsubstandard quality, it may also be important to determine whether theirrelative positions have changed as well. In other words it may benecessary to determine that the distance (or other characteristic)between Part A's authentication region and Part B's authenticationregion is effectively unchanged, and only if that is accomplished canthe weapon be authenticated. Specifications of this type can be storedwith or as part of a digital fingerprint of the firearm.

Once one or more suitable digital fingerprints of an object areacquired, the object (actually some description of it) and correspondingfingerprint may be stored or “registered” in a database. For example, insome embodiments, the fingerprint may comprise one or more featurevectors. The database should be secure. In some embodiments, a unique IDalso may be assigned to an object. An ID may be a convenient index insome applications. However, it is not essential, as a digitalfingerprint itself can serve as a key for searching a database. In otherwords, by identifying an object by the unique features andcharacteristics of the object itself, arbitrary identifiers, labels,tags, etc. are unnecessary and, as noted, inherently unreliable.

FIG. 1 is a simplified flow diagram illustrating a method 100 forcreating and storing or “registering” a digital fingerprint of an objectin a database. The process in one embodiment includes acquiring adigital image of the object, block 102, as discussed above. A variety ofimage capture technologies and devices may be used as noted. Next,features are extracted, block 104, from the digital image data. Asexplained, specific features or regions of interest (authenticationregions) may be selected in support of subsequent identification orauthentication of the object. The extracted features are analyzed andfeature vectors are extracted to form a digital fingerprint—a digitalfile or record associated with the original image data, indicated atblock 106. The digital fingerprint preferably may be stored in adatabase record at block 108. Other forms of searchable digital datastorage should be deemed equivalents. Further, at block 110,initialization data should be added to the database record, orassociated with it in a related table. This data is associated with thephysical object that was scanned. For example, a description,manufacturer, model number, serial number, contents—a wide variety ofdata, selected as appropriate or useful depending on the type of object.

FIG. 1B illustrates a process that includes more robust featureextraction. In this example, we again begin with acquiring digital imagedata, block 120. We select at least one authentication region, block122. This may be done by analysis of the image data, analysis of relatedimage data, by reference to a predetermined template that defines atleast one authentication region, or other means. The next block 124calls for extracting a feature vector from the selected authenticationregion. A feature vector may be used to represent features of a regionin a more compact form. For example, a feature vector may comprise anarray of color or gray scale numeric values corresponding to areaswithin the selected authentication region. The values may each comprisea sum, average, maximum or other function of the individual values of acorresponding group of pixels forming a sub-part of the region. In someapplications, a feature vector may identify a location and shape of adistinctive aspect within a selected region. In decision 126, there maybe additional feature vectors to be extracted from the same image data.In that case, the flow returns, path 130, to repeat the featureextraction step 124. This loop 130 may repeat until all desired featurevectors are collected. Optionally, there may be another authenticationregion to process in the same image data, see decision 132. In thatcase, the outer loop 133 is traversed back to block 122 for furtherfeature extraction with respect to one or more additional authenticationregions. Then some or all of the extracted feature vectors may becombined to form a digital fingerprint, block 134, which is then stored,block 136, along with related data, block 138, as mentioned above. Theprocess returns or concludes at block 140.

A database of digital fingerprints can form the basis of a system totrack and trace the object through a supply chain, distribution network,or sales channel. A track and trace system based on digitalfingerprinting has unique advantages and provides unique capabilitiesthat are not available with track and trace systems based on traditionalmethods.

Holograms, bar codes and serial numbers can all be duplicated withvarying degrees of effort. This means that if the code or tag can beduplicated, then counterfeit objects or two objects with the sameidentifier can exist in the supply chain or distribution network. Theycan then be registered in a traditional track and trace system. All suchsystems rely on determining that the anti-counterfeit item (label,hologram, RFID tag) is legitimate, not that the item itself is.

Due to this weakness, track and trace systems based on traditionalapproaches like bar codes or serial numbers cannot prevent the resultingcorruption of the system database. A counterfeit object may bemistakenly identified as genuine, and generate a false audit trail as itis tracked through the supply chain. Two or more objects with the sameID (one genuine, one or more counterfeit) may exist at the same time.Without physically examining the objects it is impossible to tell whichitem is genuine. Even if the objects can be physically inspected,determining which (if either) is authentic may require a subject matterexpert. Once identification is made as to which object is genuine, thefalse trails must be removed from the database to restore integrity.This can be extremely difficult depending on the structure of thedatabase and the complexity of the tracking data. In some cases theobjects may not have any further contact with the track and trace system(for instance if they are purchased by a consumer), and the record willnever be identified as false, leaving the database permanentlycorrupted.

In one embodiment of the Digital Fingerprinting Track and Trace System,an item may be scanned and identified at initial manufacture.Alternatively, an item may be scanned and identified at any subsequenttime or location for entry into a tracking system. This point ofidentification preferably is done when the item is either in thepossession of its manufacturer or has been transferred by secure meansto the current holder so that its legitimacy at the point ofidentification is adequately established. Alternatively, an item may beidentified or authenticated by a subject matter expert and scanned atthat time for entry into a tracking system.

The system then identifies the object every time it is scanned again,typically at discrete steps in manufacturing, distribution, and sale.FIG. 2 is a simplified flow diagram illustrating a method 200 formatching a digital fingerprint of a target object to a database ofexisting digital fingerprints. Here, we acquire image data of a “targetobject” i.e., the object we want to identify or authenticate by findinga match in the database, see block 202. We extract features from thetarget object image data, block 204, as discussed above. Then we createa new (second) digital fingerprint based on the extracted features,block 206. The next step is querying the database, block 208, for arecord that matches the second digital fingerprint record. “Matching” inthis context may be relative to a threshold confidence level rather thana binary decision or to a match confidence level with some other object(e.g., identify an object as legitimate when it matches the reference ofa legitimate object better (e.g., considerably better) than it matchesany other object in the database). The requisite confidence level mayvary depending on the specific application. The confidence levelrequired may be varied dynamically responsive to the data and experiencewith a given system. If no “matching” record is returned, decision 210,update the second record (the digital fingerprint of the target object),block 212, to reflect that no match was found. If a match is returned,the matching record is updated to reflect the match, block 214 (forexample, it may be linked to the second record). The results may bereturned to the user. The process returns or concludes at block 216.

Typical tracking steps might include scanning at the point ofmanufacture, when packaged, when placed in inventory, when shipped, andat a retail point of purchase (upon arrival and again when sold), asillustrated in the tracking process 300 of FIG. 3. Each scan can be usedto update a remote database.

As mentioned earlier, a “scan” may refer to an image (or digital datathat defines an image) captured by a scanner, a camera, aspecially-adapted sensor array such as CCD array, a microscope, a smartphone camera, a video camera, an x-ray machine, etc. Broadly, any devicethat can sense and capture electromagnetic radiation (or any identifyinginformation, e.g., sonar etc., that has traveled through an object, orreflected off of an object, is a candidate to create a “scan” of theobject. It is critical to capture at least one native feature of theobject, which may be of an original region of the object asdistinguished from a region having a feature added to the object foridentification, such as a label, bar code, RFID tag, serial number, etc.In some cases, the native feature may of a non-original region in whichan object has been added to the physical object for identification (suchas a label). The added object may be affixed (e.g., permanently affixed)to the physical object, such as through an adhesive in the case of alabel. So long as the added object (e.g., the label) because an integralpart of the physical object, we can scan the added object to obtain adigital fingerprint and use that digital fingerprint to track thephysical object. In some embodiments, the digital fingerprintcorresponds to an original region, a non-original region (correspondingto where an object has been added for the purpose of, for instance,identification of the physical object), or combinations thereof.

A “native feature” in this description may not be concerned with readingor recognizing meaningful content, even in the case where the digitalfingerprint corresponds to a non-original region. For example, a labelon a scanned object with a printed serial number may give rise tovarious features in fingerprint processing, some of which may becomepart of a digital fingerprint feature set or vector that is associatedwith the object. The features may refer to light and dark areas,locations, spacing, ink blobs, etc. This information may refer to theprinted serial number on the label, but there is no effort to actually“read” or recognize the printed serial number (which may be bogus).Similarly, an RFID tag applied to an object may give rise to afingerprint vector responsive to its appearance and location on theobject. However, in some examples no effort is made to actuallystimulate or “read” data or signals from the tag. In some embodiments weare not using the added object according to the tracking scheme fromwhich it originated. The various features used in fingerprintprocessing, some or all of which may become part of a digitalfingerprint set or vector that is associated with the physical object,may be extracted from a permanently affixed label (for the purposespresented here the contents of the label, e.g., the value of the serialnumber) may be irrelevant.

While the most common application of track and trace systems is inmanufactured goods, the present system and methods, in various differentembodiments, may be applied to any object that can be identified with adigital fingerprint and tracked. These include but are not limited tomail pieces, parcels, art, coins, currency, precious metals, gems,jewelry, apparel, mechanical parts, consumer goods, integrated circuits,firearms, pharmaceuticals, and food and beverages. Tracking may consistof any sequence of actions where the object is scanned, such as eachtime an object is appraised, authenticated, certified, auctioned,displayed, or loaned. The system may store both positive and negativeauthentication transactions. In an embodiment, the system may storelocation information (associated with a scan or fingerprint), whichprovides a profile of where either counterfeit or legitimate goods maybe encountered.

FIG. 4 is a simplified conceptual diagram illustrating a database systemand use of a mobile device application to query authenticationinformation related to an object. Here, various computing devices orterminals 402 may have access over a network, for example, the Internet404, to cloud computing facilities/services such as a cloudserver/datastore 406. For example, the devices 402 may be located atvarious points along a distribution chain as illustrated in FIG. 3, eachlocation scanning an object and updating the cloud server/datastore 406.

In some embodiments, the mobile device application may acquire imagedata of at least a portion of a target physical object. The mobiledevice application may utilize a camera integrated or coupled to thedevices 402 to acquire the image data, in some examples. The mobiledevice application may analyze the image data to form a digitalfingerprint. The mobile device application may query a datastore, suchas the datastore 420, to seek a matching record based on the digitalfingerprint. The mobile device application may update a database recordof the datastore 420 with an indication that the digital fingerprint wasmatched to the database record.

A server 412 may be provisioned to provide tracking and/or tracing dataanalysis and reporting. The server 412 has access to a datastore 420which may be used to store digital fingerprints and related data. Theserver can query or search the database 420 for digital fingerprintsearch and matching. The database 420 preferably is coupled to the cloudserver 406 in some embodiments. A mobile user device 410 such as asmartphone, tablet, laptop computer or dedicated device may beconfigured for communications with the server 412 to request and receivea reply or authentication report for an object of interest. Thisarchitecture is simplified and in any event is merely illustrative andnot intended to be limiting.

Continuous and Discrete Tracking

In some implementations, sensors may be attached to the object, andsensor data can flow back to the database in either a continuous fashion(near real time), or in discrete data transfer events. For example, datatransfer may occur when an authentication event occurs. For instance, ifthere is a GPS chip attached to the object, data flow can start when theobject is first registered in the system, and continue to flow as theobject changes location. Continuous (frequent) data updates can also bebuffered in local memory in a sensor attached to the item, and thendownloaded the next time the object is scanned and authenticated. Thisprovides a record of where the object has traveled (its itinerary).

As an example of the potential uses of sensor data, many products likefood and beverages can degrade with exposure to certain environmentalfactors over the course of their storage and shipment. Examples ofsensor data could include temperature, light exposure, altitude, oxygenlevel, or other factors, as well as location such as GPS data.

FIG. 5 is a simplified flow diagram illustrating one embodiment of aprocess 500 for tracking an object to verify its provenance. Here, anexpected itinerary of an object (a series of locations) may be stored ina datastore if known, block 502. The methods and systems described abovemay be used to track the object to the next location, block 504. If theobject does not arrive as expected (where and when expected according tothe itinerary), the failure may be reported to a user. In an embodiment,an object that arrives significantly earlier or later than expected maybe subjected to closer matching scrutiny to ensure its identity.

The next step, block 510, is to query the database for the next valid orexpected location. A unique itinerary may not be known, but a set ofvalid or expected locations may be known. The next actual location ofthe object (as determined by imaging and matching digital fingerprints)may be compared to the expected location(s) returned by the database,block 512. If that comparison indicates a departure from the expected orauthorized route, decision 520, the result may be reported to a user,block 522. (A report that the object is on track may be reported aswell.) Other options may be implemented such as a quantity check, block524. The process returns or terminates at block 526.

Most existing track and trace systems are only designed to be accessedby manufacturers or their authorized distributors, and often requirespecialized scanners or equipment. However, the consumer also has avested interest in determining whether the items that they are buyingare authentic or to determine where they are. In some embodiments, thepresent system is designed to enable anyone along the supply,distribution, or sales chain, from manufacturer to the retail consumer,to access the system and determine whether the item is authentic andwhere it is (e.g., a current location when it is being shipped toconsumer). A specialized scanner is not required in all cases. Forexample, in one embodiment a mobile phone application designed for theconsumer can be used to scan an object, query the database, anddetermine if the object is authentic and retrieve current data about it.

Finally, data collected by a digital fingerprinting system offers avariety of useful information to people along the supply, distributionand sales chain. Reports can be generated on individual items, or onsets of items. These reports can include but are not limited to thelocations of items over time, audit trails, points of entry ofcounterfeit goods, and exposure to environmental variables over thecourse of an object's lifetime.

Tags and Bar Codes

A tag may be added to an item, a barcode to a mail piece, etc. for tworeasons. First, the human may need it to know what the item is. This isthe identification function. It may identify the item to a store clerkas a particular style and size of clothing of a particular manufacturer,it may tell a postal carrier where to deliver a mail piece. Second,however, are tags that are only useful for a machine. Thus a four-statebar code on a mail piece (unreadable by humans) is used to route themail piece by machine. This entire class of machine readable tags can bereplaced by the methods of this patent. The first set may still beneeded for human use but are now divorced from their authenticationfunction.

Because we are exploiting natural features and often scanning the objectunder variable conditions, it is highly unlikely that two different“reads” will produce the exact same fingerprint. We therefore have tointroduce the ability to look up items in the database when there is anear-miss. For example, two feature vectors [0, 1, 5, 5, 6, 8] and [0,1, 6, 5, 6, 8] are not identical but (given the proper differencemetric) may be close enough to say with certainty that they are from thesame item that has been seen before. This is particularly true if,otherwise, the nearest feature vector of a different item is [5, 2, 5,8, 6, 4]. For example, a distance between vectors of n-dimensions iseasily calculated, and may be used as one metric of similarity or“closeness of match” between the vectors. One may also consider thedistance to the next nearest candidate.

Obviating the Chain of Custody

Many systems rely on a known “chain of custody” to verify authenticityof an object. The rules of evidence in court, for example, typicallyrequire proof of a chain of custody to demonstrate authenticity of ahair sample, weapon or other piece of physical evidence. From the timean object is collected at a crime scene, for example, it is typicallybagged, tagged, and moved into a locked box or evidence room forsafekeeping. Each person who removes it must attest to returning theoriginal item unchanged. Custody of the object from the crime scene tothe evidence locker to the courtroom must be accounted for as anunbroken chain. Digital fingerprinting techniques as disclosed hereincan be used to obviate most of that process. Provided an original objectis under the control and custody of an authorized or trusted entity atleast once, and a digital fingerprint, or an image suitable for forminga digital fingerprint of the object, is acquired under suchcircumstances and stored, the object is uniquely identifiable thereafterby that fingerprint for the lifetime of the object.

Because digital fingerprinting works by extracting key features of anobject, it may be used to identify or authenticate objects even after agood deal of wear and tear. At any subsequent time, a suspect or“target” object can be similarly “fingerprinted” and the subsequentfingerprint compared to the stored fingerprint of the original object.If they match, authenticity is established, regardless of where or inwhose custody the object may have traveled in the meantime. Returning tothe crime scene example, if a digital fingerprint is acquired of aweapon taken from the crime scene, and the digital fingerprint stored,the weapon can be fingerprinted again at any subsequent time, and thedigital fingerprints compared to authenticate to weapon. Custody of theweapon in the interim is no longer an issue. Likewise when a coin orpiece of art is stolen, our technology gives us the ability to continuethe original provenance even though chain of custody has been lost (i.e.we know it is the same coin we saw before even though it has not been inour direct possession during the time between theft and recovery).

Global Vs. Regional Feature Matching

In a case where we have the original document or other objectfingerprinted, our techniques allow region-by-region matching so that wecan tell what (if any) regions have been changed. Thus, for example, wemight get a really good overall match on a passport but none of thematches happen in the photograph—so we know the photograph probably waschanged. Further, if some individual or group, say Al Qaeda, has acertain pattern or regularity to altering passports—change the photo,the date of birth and one digit of the passport number, say—then thisability to find altered regions also gives us the ability to discern thepattern of changes and thus develop a signature of the group making thechanges. Thus aspects of the present technology can be applied not onlyto detect a forged or altered document, but to identify in some casesthe source of the bogus document.

Multi-Level Authentication

Some systems using digital fingerprinting work by convertingnaturally-occurring features (or features that occur as a side effect ofthe manufacturing process) on an item into a vector that is stored in adatabase until the object is later presented for authentication, invarious embodiments. At that point a new digital fingerprint may beextracted and compared with the digital fingerprint(s) in the database.When sufficient match is found, the item is considered authenticated.

Items can be tracked and authenticated throughout their life cycle asthey are produced, aggregated into, say, boxes of parts (some examplesmay utilize multiple levels of packaging), shipped while aggregated,broken into individual items, assembled into, say products, and finallymade available to the final consumer. Items may be tracked at all levelswhether or not they are temporarily or permanently part of someaggregate. Thus not only the final item received by the consumer butalso all the intermediate aggregates and components can be tracked as acoherent whole, with each component stream trackable in its own rightand as parts of the whole.

Some embodiments may utilize fingerprint-based authentication formulti-layered authentication; however, this is not required. Multi-levelauthentication may be applied to any form of authentication in which:

-   -   There are multiple levels of packing, assembling, disassembling,        or otherwise layering the items; and    -   Any history, tracking, or authentication done on each level is        considered appropriate also for the level below, e.g., for the        items inside the packaging.

In some embodiments in which any history, tracking, or authenticationdone on each level is considered appropriate also for the level below,some embodiments may include a scheme to ensure that the contents of thepackage have not been tampered with and that items within the higherlayer may be authenticated upon entry and/or upon removal from thelayer.

In some embodiments, a network of track and trace and authenticationdata may be built, stored in a database, and accessed where appropriatefor all the components, subsystems, and complete parts in a system. Suchembodiments may be utilized for items that by their nature areaggregated at some point in their life cycle (e.g., wine, electronicdevices, etc.).

Wine may be digitally fingerprinted (or otherwise set up for trackingand/or authentication) when bottled, then the case into which it isshipped be separately authenticated. Finally the containers into whichthe cases are placed for shipping can also be digitally fingerprinted.The shipping container may include a tamper-proof seal so that one maybe sure that the contents of a shipping container, for example, are notsubstituted or removed during the period the wine cases are supposed tobe inside. During the period of aggregation, each bottle is tracked ifthe case is tracked; each case is tracked if the container is tracked,and the history of the upper level becomes the history of the lower.This may include track and trace information and/or other possiblemetadata such as temperature measurements, acceleration measurements,pressure (e.g., atmospheric pressure) measurements, humiditymeasurements, chain of ownership, location, or the like, or combinationsthereof.

In some embodiments, component and/or system tracking may be used. Asystem may include an electronic device for use in a critical system inan airplane, which may include components such as chips. Detection ofsubstitution of the correct chips with lower-performance chips may be aconcern. In some embodiments, chips may be digitally fingerprinted atmanufacture and each may be given history data (which may include allmetadata) and a set of digital fingerprints (or a respective one of aset of digital fingerprints). The history data may be used to establishthat the history for each chip is acceptable to the final consumerand/or a regulatory body, while the digital fingerprints may be used toauthenticate the appropriate parts where needed.

The chips may be shipped to a company that assembles them into systems,such as processing cards. A process used for the assembly of the cardsand/or a nature of the processing cards may preclude undetectableremoval of parts (e.g., chips) from the processing cards. Accordingly,the processing cards may become the primary trackable item going forward(for critical systems the individual chips may continue to beauthenticated). Later the processing cards may be put into (e.g.,installed) a final part which may itself be digitally fingerprinted andshipped to the final user where, if the final part is secure fromtampering, authenticating the final part also authenticates all items inthe part manifest.

FIG. 7 illustrates a process 700 of re-authenticating an object based onauthenticating an aggregate.

The process 700 can be used in digital fingerprinting embodiments, orany embodiments using a database system used for tracking and/orauthenticating multilevel items (such as wine bottles in wine cases,chips on processing cards, etc.). In the case of digital fingerprintingembodiments, a physical object (such as a wine bottle or chip) may bedigitally fingerprinted using the devices of FIG. 4 in association withforming an aggregate physical object (such as a wine case including thewine bottle or a processing card including the chip). Later, such as ata different location in a supply chain, the aggregate physical objectmay be authenticated using the devices of FIG. 4. An indication of are-authentication of the physical object may be stored in the databasesystem. In some embodiments, additional data associated with theauthentication of the aggregate physical object may also be stored inthe database. The additional data may include metadata about theauthentication (location, confidence, etc.) and/or data collected inassociation with the authentication, such as a temperature measurement,a reading from a sensor, etc.

In some embodiments, the indication of re-authentication may be storedin one or more first database record(s) different than the seconddatabase record for the aggregate object. The one or more first databaserecord(s) may be linked to the second database record.

Besides digital fingerprinting embodiments, it is also possible toauthenticate the aggregate object using any other authentication scheme.For instance, a value from an object added to the aggregate physicalobject for the purpose of authentication and/or tracking could be readat the different location, validated, and an indication ofre-authentication could be stored in the database system responsive toresult of the validation of the value. The processes of multilevelauthentication described herein, such as the process 700 and a process800 (FIG. 8), are not limited to digital fingerprinting. Regardless ofthe type of authentication used (digital fingerprinting or otherwise),other information besides a binary result (e.g., authenticated, or not)may also be stored in the database (e.g., in a database record for theaggregate and/or database records for the individual objects) responsiveto the result of the validation of the value (e.g., information about aconfidence of the indication of re-authentication may also be stored inthe database, authentication information such as a digital fingerprintor value associated with the physical object, or the like, orcombinations thereof).

In block 702, a database system (e.g., the database system of FIG. 4)may initialize database records of individual physical objects. In block704, an aggregate physical object may be created from the individualobjects (e.g., a processing card may be manufactured by fixably couplinga chip to a processing card, a bottle of wine may be loaded into a case,etc.). In block 706, the database system may initialize a databaserecord for the aggregate physical object. The database record may be adifferent record that is linked to the database records for theindividual objects, in some examples.

In block 708, object authentication may be performed, for example theindividual objects may be authenticated in association with creation ofthe aggregate physical object. In some examples, the authentication maybe by capturing image data from the bottle of wine, a chip, etc.,forming a digital fingerprint, and so on, but this is not required. Anyscheme for authenticating the bottle of wine may be utilized, in someembodiments. In block 710, the database system may retain an indicationof authentication of the individual physical objects and/or metadata. Insome examples, blocks 702-710 may include the database system performingthe process shown in FIG. 1A for the individual objects and theaggregate physical object.

At a different time and/or location (for example at a next location in asupply chain) a target physical object may be obtained. In block 720,the database system may query the database for a matching record. Insome embodiments, the database system may perform fingerprint matching200 (FIG. 2) for the target physical object.

In diamond 722, the database system may determine whether a record ofthe aggregate physical object is returned responsive to the query. Ifthe record of the aggregate physical object is returned, then in block724 the database system may update the database system to reflect thatdata of the target object was matched, which may be similar to block 214(FIG. 2) in some examples. The database system may retain an additionalindication of authentication (e.g., an indication of re-authentication)of the individual objects and/or metadata (from the authentication ofthe aggregate physical object). In the case that the record of theaggregate physical object is not returned, in block 726 the databasesystem may update the database to reflect that data of the target objectwas not matched, which may be similar to block 212 (FIG. 2) in someexamples.

FIG. 8 illustrates a process 800 of tracking objects and aggregatesthereof. An object may be tracked along an initial leg of a supplychain, assembled into an aggregate, and then the aggregate may betracked along another leg of the supply chain.

In block 802, the database system may store data for known locations foran object. This may be similar to block 502 of FIG. 5. The object may bean individual object, such as a wine bottle or chip, that is to be usedto create an aggregate object, such as a wine case, a shipping containfor wine cases, a processing card, etc. The database system may performblocks 854, 860, 862, 870, 872, 874, and 856, which may be similar toblocks/diamonds 504, 510, 512, 520, 522, 524, and 506, respectively.

In diamond 804, the database system may determine whether another objectis to be tracked. If another object is to be tracked, in block 806 anaggregate may be created from the object (wine may be loaded into acase, a chip may be installed on a card, etc.). In block 808, thedatabase system may identify stored data for known locations for theaggregate. In some examples, this may include accessing data stored forthe individual object, in some cases. In block 810, the database systemmay associate the stored data and/or store new data (e.g., data createdfor the aggregate) for known locations of the aggregate. Tracking forthe aggregate may be performed as indicated by the return to block 854.Also, the wine case may be packaged into another aggregate (such as ashipping container) as indicated by the return to diamond 804.

Example Embodiments

Example A-1 is a method of: authenticating a physical object of aplurality of physical objects that together form an aggregate physicalobject, wherein the aggregate physical object comprises either anintermediate aggregate physical object from which said physical objectis to be later separated or a final aggregate physical object; storingin a database system relationship information reflecting a relationshipbetween the aggregate physical object and the plurality of physicalobjects; attempting to authenticate a target physical object; responsiveto matching the target physical object to the aggregate physical objectbased on the attempt to authenticate the physical target: identifying inthe database system a database record corresponding to the aggregatephysical object; storing in the database record authentication datareflecting the match between the target physical object and theaggregate physical object; and storing an indication of are-authentication of the physical object in the database system based onthe relationship information.

Example A-2 may include the subject matter of example A-1 (and/or anyother example herein), wherein authenticating the physical objectcomprises: acquiring digital image data of an image of at least aportion of the physical object; analyzing the digital image data to forma digital fingerprint, wherein the digital fingerprint is responsive tonatural structure of the physical object; and storing the digitalfingerprint in the database system.

Example A-3 may include the subject matter of example A-1 (and/or anyother example herein), wherein attempting to authenticate the targetphysical object comprises: acquiring first digital image data of animage of at least a portion of the aggregate physical object; analyzingthe first digital image data to form a first digital fingerprint,wherein the first digital fingerprint is responsive to natural structureof the aggregate physical object; storing the first digital fingerprintin the database record; subsequent to storing the first digitalfingerprint in the database record, acquiring second digital image dataof an image of at least a portion of the target physical object;extracting features from the second digital image data to form a seconddigital fingerprint; and querying the database system to seek a matchingrecord based on the second digital fingerprint.

Example A-4 may include the subject matter of example A-3 (and/or anyother example herein), wherein attempting to authenticate the targetphysical object further comprises: in the case that the database recordis returned responsive to the querying, updating the database recordwith an indication that the second digital fingerprint was matched tothe database record.

Example A-5 may include the subject matter of example A-1 (and/or anyother example herein), further comprising storing the indication in adifferent database record of the database system, wherein the differentdatabase record is linked to the database record.

Example A-6 may include the subject matter of example A-1 (and/or anyother example herein), wherein the plurality of physical objectscomprise bottles of wine, and the aggregate physical object comprises acase containing the bottles of wine.

Example A-7 may include the subject matter of example A-1 (and/or anyother example herein), wherein the plurality of physical objectscomprise first bottles of wine, second bottles of wine, a first case forthe first bottles of wine, a second case for the second bottles of wine,and a shipping container for the first and second cases; and wherein theaggregate physical object comprises the shipping container including thefirst and second cases each including the first bottles of wine and thesecond bottles of wine, respectively.

Example A-8 may include the subject matter of example A-7 (and/or anyother example herein), wherein the physical object comprises a singleone of the bottles of wine.

Example A-9 may include the subject matter of example A-7 (and/or anyother example herein), wherein the physical object comprises a singleone of the cases.

Example A-10 may include the subject matter of example A-1 (and/or anyother example herein), wherein the plurality of physical objectscomprises a microchip and a printed circuit board, and the aggregatephysical object comprises a processing card, wherein the microchip isinstalled on a printed circuit board of the processing card.

Example A-11 may include the subject matter of example A-3 (and/or anyother example herein), further comprising selecting the portion of theaggregate physical object to correspond with a portion of theintermediate aggregate physical object that is to be opened ormanipulated to separate the physical object from the intermediateaggregate.

Example A-12 may include the subject matter of example A-3 (and/or anyother example herein), further comprising repeating said authenticationof the physical object in association with separating the physicalobject from the intermediate aggregate.

Example A-13 may include the subject matter of example A-1 (and/or anyother example herein), further comprising for the physical object,defining an expected itinerary along a supply chain; tracking theaggregate physical object along the supply chain to form a portion of anactual itinerary of the physical object in the database system; queryingthe database system to detect a departure of the actual itinerary fromthe expected itinerary; and reporting the detected departure to a user.

Example A-14 may include the subject matter of example A-13 (and/or anyother example herein), wherein the departure comprises a failure tolocate the aggregate physical object at an expected location along theexpected itinerary.

Example A-15 may include the subject matter of example A-13 (and/or anyother example herein), wherein the departure comprises locating eitherthe aggregate physical object or one of the plurality of physicalobjects at a location that is not consistent with the expecteditinerary.

Example A-16 may include the subject matter of example A-1 (and/or anyother example herein), further comprising obtaining additional data forthe aggregate physical object and storing information about theadditional data in the database record.

Example A-17 may include the subject matter of example A-16 (and/or anyother example herein), wherein the additional data comprises temperaturedata, acceleration data, humidity data, pressure data, or anothermeasurement.

Example A-18 may include the subject matter of example A-16 (and/or anyother example herein), wherein the additional data comprises locationdata.

Example A-19 may include the subject matter of example A-16 (and/or anyother example herein), wherein the additional data comprises chain ofownership data.

Example A-20 may include the subject matter of example A-3 (and/or anyother example herein), wherein the digital fingerprint does not relyupon or require any labels, tags, integrated materials, uniqueidentification characters, codes or other items that were added to thephysical object specifically for the purpose of identification.

Example A-21 may include the subject matter of example A-1 (and/or anyother example herein), wherein the authenticating is associated withcreation of the aggregate physical object.

Example A-22 is a database system, wherein one or more processors of thedatabase system are configured to perform the steps of any one ofexamples A-1 through A-21.

Example A-23 may include the subject matter of example A-22 (and/or anyother example herein), wherein a first one of the one or more processorsoperates in a mobile device of the database system, and wherein thefirst processor is configured to perform the steps of A-2 and A-3, orany of the other example of examples A-1 through A-21 (and/or any otherexample herein).

Example A-24 is one or more memory storing instructions to be executedby one or more processors, wherein the instructions when executedperform operations corresponding to the steps of any of the examples A-1through A-21 (and/or any other example herein).

Example A-25 may include the subject matter of example A-24 (and/or anyother example herein), wherein at least a portion of the one or moreprocessors comprises a processor of a mobile device.

Personal History in Track and Trace System

It may be of great interest or importance to an entity (a manufacturer,an ultimate consumer of an item, a regulatory body, or the like) to knowan item's history. Some embodiments of a track and trace system mayenable ascertainment that the item being authenticated at some point inthe distribution process is indeed the item that was produced. This mayaddress important need of a manufacturer, a consumer, etc.: providingknowledge that the consumed item is the item that was produced. Repeatedauthentication and/or data collection may be enforced throughout thesupply chain by a manufacturer, a consumer, etc. An airline may, forexample, require its manufacturers to digitally fingerprint an item forlater authentication and also require each intermediate distributor todo the same until the part is ultimately installed on an airplane. Suchrequirements are not required for creating a personal history for anitem, but they may be synergistic with creating a personal history foran item.

In some applications, simply knowing that an item is the original is notenough (e.g., the life history of the item may be critical as well).Items such as food, wine, computer chips, etc., may require being keptwithin a particular temperature range to be useful and/or retain theirfull value at their ultimate point of consumption. A wine that has beenexposed to tropic heat, for example, has lost its value and it is oflittle consolation that the consumer knows the ruined wine is still inits original bottle.

There are many items, wine important among them, that are part of aceremonial or almost ritual process. Both collectors of wine andconsumers of wine want to know where the wine has been, who has ownedit, and how it has been maintained. A rich pedigree adds to theenjoyment of a good bottle of wine. Some embodiments disclosed hereinmay allow that pedigree to be presented to the consumer by, say, thesommelier as part of the pouring process.

Some embodiments may establish and record the personal history of anitem from its point of manufacture/creation (or point of establishmentof preservation of a level of authenticity of a physical object) to itspoint of consumption.

In the process of tracking items using embodiments disclosed herein, theitem and/or an intermediate or final aggregate item that includes theitem may be digitally fingerprinted many times. A personal history forthe item(s) may be established by tying that digital fingerprinting tometadata about the item(s). A part, such as an electronic apparatus,which is not subjectable to great acceleration without risk of damage,may have an accelerometer attached prior to shipment and a method ofreading the history of shock loads experienced by the item provided atthe point of authentication. At some or all points of authenticationand/or data collection during track and trace of the item, along withcollecting and comparing digital fingerprints, the acceleration historyof the item may be read to create metadata that may be tied to track andtrace record(s) for the item(s).

Knowing that such tracking is to be performed may provide incentive formanufacturers to include items (such as a component that is notsubjectable to great acceleration without risk of damage) with theirproduct (since when coupled with a track and trace system it can beestablished which distributor (for example) dropped the item, causing itto experience out-of-tolerance acceleration).

Among the data that might be acquired and added to the tracking recordare GPS (global positioning system) or other location information, chainof ownership, acceleration, temperature, point of assembly or division(e.g. where a case of medicine was broken into individual packages or aproduct was assembled from its component parts), or the like, orcombinations thereof. Any type of process for reading this informationmay be used, such as RFID (radio frequency identification) systems that,when queried, read out the history. For some data, manual entry may besufficient (e.g. chain of ownership) alone or in combination withreading by electronic systems such as RFID based systems. Someembodiments may include track and trace capabilities to create themetadata records for preserving such information, as well as accesscapabilities enabling the metadata records to later be used.

As mentioned above, personal history may not be limited to the itemitself, but may also include “parentage”. For example, a case of winemay have been tracked and every bottle inside may have been digitallyfingerprinted at manufacture. The same may be true for many other items,including packages of pharmaceuticals. Up to the point where theintermediate aggregate item (e.g., the case of wine) is opened, everyitem inside the intermediate aggregate item has the same personalhistory. At the point where the intermediate aggregate item is openedand the items thereof (e.g., the bottles of wine) distributedindividually, each item separable from the intermediate aggregate item(e.g., a bottle of wine, a pill bottle, an individual part etc.) mayacquire by inheritance the history of its “parent”, namely theintermediate aggregate item (e.g., the case of wine) from which it came.Thereafter, the separate items may have their own history.

Some embodiments to create a personal history for an item may operate ina same system of an embodiment utilizing multi-level authentication.Creating a personal history may include establishing a history ofindividual items and of the different levels of packaging those itemsare, at one point or another in the item's life cycle, transported in.It is in view that items may, for example, be packaged into intermediateaggregate items (such as cases of wine) at the point of manufacture andlater further transported individually. Conversely, it is also in viewthat multiple items may be aggregated into a single shipment orassembled into a product and thereafter (for a while or until finalconsumption) shipped as an aggregate.

In some systems, general information may be available for batches of anitem but not for individual units. For example, batch numbers may beused to track, say, pharmaceutical aggregates. Similarly, for wine,information may be available for wine for the vineyard, type of grape,and vintage, but not for each bottle. In the case of wine, people sellwine notes concerning the vineyards, grape types, and vintages for thewine, but the only knowledge of a particular bottle comes from a bottlethat has, by definition, been consumed. Some embodiments disclosedherein may allow information to be gathered on each individual bottleand traced through the entire lifespan of that bottle. Some embodimentsmay realize both item-level tracking and creation of item-levelhistories.

FIG. 9 illustrates a process 900 of creating a personal history forcomponents of a system, such as chips of a processing card.

In block 902, a database system (such as the database system of FIG. 4)may identify a matching record in a database for a target object. Insome examples, this may be similar to process 200 (FIG. 2). In block904, data for the target object may be obtained. For example, a sensorof the target object may be read, the target object may be measured,etc.

In diamond 906 the database system may determine whether a matchingrecord is for a system. If the matching record is for a system, thedatabase system in block 910 may add the obtained data to first storeddata for components of the system to create personal history data forthe components. The database system may also add the obtained data tosecond stored data to create personal history data for the system inblock 912.

If the matching record is not for a system in diamond 906 and the matchis to a record for a discrete object (block 920), then in block 922 thedatabase system may add the obtained data to third stored data for thediscrete object to create personal history data for the discrete object.

FIG. 10 illustrates a process 1000 of creating a personal history forobjects of a supply chain aggregation, such as wine bottles of a winecase and/or shipping container.

In block 1002, a database system (such as the database system of FIG. 4)may identify a matching record in a database for a target object. Insome examples, this may be similar to process 200 (FIG. 2). In block1004, data for the target object may be obtained. For example, a sensorof the target object may be read, the target object may be measured,etc.

In diamond 1006 the database system may determine whether a matchingrecord is for an intermediate aggregate (such as a case of wine wherethe wine is to be later separated from the case). If the matching recordis for an intermediate aggregate, the database system in block 1008 mayadd the obtained data to first stored data for individual items to beseparated from the intermediate aggregate to create personal historydata for the individual items. The database system may also add theobtained data to second stored data to create personal history data forthe intermediate aggregate in block 1012.

If the matching record is not for a system in diamond 1006 and the matchis to a record for a discrete object (block 1020), then in block 1022the database system may add the obtained data to third stored data forthe discrete object to create personal history data for the discreteobject.

Example Embodiments

Example B-1 is method, comprising: acquiring first digital image data ofan image of at least a portion of a physical object; analyzing the firstimage data to form a first digital fingerprint of the physical object,wherein the digital fingerprint is responsive to natural structure ofthe physical object; storing the digital fingerprint in a databaserecord of a database system; subsequent to storing the digitalfingerprint in the database record, identifying first informationcorresponding to a target physical object, wherein identifying the firstinformation corresponding to the target physical object includesacquiring second digital image data of an image of at least a portion ofthe target physical object; identifying second information correspondingto the target physical object; extracting features from the seconddigital image data to form a second digital fingerprint; querying thedatabase system to seek a matching record based on the second digitalfingerprint in the case that a matching record is returned responsive tothe querying, updating the matching record with an indication that thesecond digital fingerprint was matched to it and updating history dataof said matching record based on said second information.

Example B-2 may include the subject matter of example B-1 (and/or anyother example herein), wherein said second information comprises ameasurement of a selected metric associated with the target physicalobject.

Example B-3 may include the subject matter of example B-2 (and/or anyother example herein), wherein the second information comprises atemperature value or a measurement generated by an accelerometer.

Example B-4 may include the subject matter of example B-1 (and/or anyother example herein), wherein the second information comprises locationinformation.

Example B-5 may include the subject matter of example B-4 (and/or anyother example herein), wherein the location information comprises GPS(global positioning system) of a location of the target physical objectat a time associated with the acquisition of the second digital imagedata.

Example B-6 may include the subject matter of example B-1 (and/or anyother example herein), wherein the second information comprises chain ofownership information.

Example B-7 may include the subject matter of example B-1 (and/or anyother example herein), wherein the second information comprisesinformation about a point of assembly of plural individual objects toform the target physical object or a point of division of the targetphysical object to separate an individual object from the targetphysical object.

Example B-8 may include the subject matter of example B-1 (and/or anyother example herein), wherein the second information comprises aplurality of measurements obtained over time.

Example B-9 may include the subject matter of example B-1 (and/or anyother example herein), wherein identifying the second informationcomprises reading sensor data of a sensor coupled to the target physicalobject.

Example B-10 may include the subject matter of example B-1 (and/or anyother example herein), further comprising: securely associating a sensorand the physical object; obtaining a measurement of the physical objectusing the sensor; storing a value corresponding to the measurement inthe digital fingerprint in the database record; and controlling thesensor to cause the sensor to obtain additional measurements of thephysical object periodically.

Example B-11 may include the subject matter of example B-10 (and/or anyother example herein), wherein the second information comprises datagenerated by the sensor responsive to periodically obtaining theadditional measurements.

Example B-12 may include the subject matter of example B-1 (and/or anyother example herein), wherein the digital fingerprint does not relyupon or require any labels, tags, integrated materials, uniqueidentification characters, codes or other items that were added to thephysical object specifically for the purpose of identification.

Example B-13 may include the subject matter of example B-1 (and/or anyother example herein), further comprising: in the case that a matchingrecord is not returned; updating the matching record with an indicationthat the second digital fingerprint was not matched to it.

Example B-14 may include the subject matter of example B-1 (and/or anyother example herein), wherein the matching recording comprises thedatabase record or a record of an intermediate aggregate physical objectfrom which said physical object is to be later separated or a finalaggregate physical object that includes the physical object.

Example B-15 may include the subject matter of example B-14 (and/or anyother example herein), further comprising updating history data of therecord of the intermediate aggregate physical object or the finalaggregate physical object based on the second information.

Example B-16 is a method, comprising: acquiring first informationcorresponding to a physical object (e.g., obtaining data from an objectsuch as a label or RF ID tag added to a physical object for the purposeof identification of the physical object); identifying firstauthentication data based on the first information (e.g., recognize avalue in the information from, say, the label or the RF ID tag, thevalue to be used for authentication, form a digital fingerprint, or thelike, or combinations thereof); storing the first authentication data ina database record of a database system; subsequent to storing the firstauthentication data in the database record, acquiring second informationcorresponding to a target physical object and identifying secondauthentication data based on the second information (e.g., identifying avalue in a label, RF ID tag, etc.); identifying third informationcorresponding to the target physical object; querying the databasesystem to seek a matching record based on the second authenticationdata; in the case that a matching record is returned responsive to thequerying, updating the matching record with an indication that thesecond authentication data was matched to it and updating history dataof said matching record based on said third information.

Example B-17 may include the subject matter of example B-16 (and/or anyother example herein), wherein the third information comprises at leastone of a measurement of a selected metric associated with the targetphysical object, a temperature value or a measurement generated by anaccelerometer, first location information, second location informationcomprising comprises GPS (global positioning system) of a location ofthe target physical object at a time associated with the acquisition ofthe second information, chain of ownership information, informationabout a point of assembly of plural individual objects to form thetarget physical object or a point of division of the target physicalobject to separate an individual object from the target physical object,a first plurality of measurements obtained over time, or a secondplurality of measurements obtained over time by reading sensor data of asensor coupled to the target physical object.

Example B-18 may include the subject matter of example B-17 (and/or anyother example herein), further comprising: securely associating a sensorand the physical object; obtaining a measurement of the physical objectusing the sensor; storing a value corresponding to the measurement inthe database record and/or in the first authentication data in thedatabase record; and controlling the sensor to cause the sensor toobtain additional measurements of the physical object periodically.

Example B-19 may include the subject matter of example B-18 (and/or anyother example herein), wherein the third information comprises datagenerated by the sensor responsive to periodically obtaining theadditional measurements.

Example B-20 may include the subject matter of example B-16 (and/or anyother example herein), wherein the first authentication information doesnot rely upon or require any labels, tags, integrated materials, uniqueidentification characters, codes or other items that were added to thephysical object specifically for the purpose of identification.

Example B-21 may include the subject matter of example B-16 (and/or anyother example herein), further comprising: in the case that a matchingrecord is not returned; updating the matching record with an indicationthat the second authentication data was not matched to it.

Example B-22 may include the subject matter of example B-16 (and/or anyother example herein), wherein the matching recording comprises thedatabase record or a record of an intermediate aggregate physical objectfrom which said physical object is to be later separated or a finalaggregate physical object that includes the physical object.

Example B-23 may include the subject matter of example B-22 (and/or anyother example herein), further comprising updating history data of therecord of the intermediate aggregate physical object or the finalaggregate physical object based on the third information.

Example B-24 is database system, wherein one or more processors of thedatabase system are configured to perform the steps of any one ofexamples B-1 through B-23 (and/or any other example herein).

Example B-25 may include the subject matter of example B-24 (and/or anyother example herein), wherein a first one of the one or more processorsoperates in a mobile device of the database system, and wherein thefirst processor is configured to perform the steps of B-16, or any ofthe other example of examples B-1 through B-23 (and/or any other exampleherein).

Example B-26 is one or more memory storing instructions to be executedby one or more processors, wherein the instructions when executedperform operations corresponding to the steps of any of the examples B-1through B-23 (and/or any other example herein).

Example B-27 may include the subject matter of example B-26 (and/or anyother example herein), wherein at least a portion of the one or moreprocessors comprises a processor of a mobile device.

Preserving a Level of Confidence of Authenticity of an Object

An item may be digitally fingerprinted at the point of manufacture andtracked with repeated digital fingerprinting (of the item or anintermediate or final aggregate) and database comparisons until ultimateuse of the item. Preservation of the level of confidence of authenticitymay apply to items already in existence prior to the ability to trackthem by using digital fingerprinting or a similar scheme. These itemsmay have been created and, for example, bar coded or had some securitydevice added to it (such as a security device presumed to reducecounterfeiting), or they may currently be in the hands of an expert whovalidates the item as legitimate (such as in the case of fine art or oldwine). Other examples may include currently-valid passports and otheralready-existing identity documents that have some existing level oftrust.

Some embodiments utilizing preservation of the level of confidence ofauthenticity may use digital fingerprinting to preserve the level ofprovenance established by an authentication framework (whether expert,machine-based, or a combination thereof) that is current when the itemis about to be digitally fingerprinted.

Consider a wine collector who hires an expert authenticator to certifythat the items in his collection are what they claim to be. It may notbe possible to establish without question the original provenance of theitem (e.g. tracing back to a point of manufacture). Similarly, the levelof confidence of authenticity currently present may not have beendirectly established at the point of manufacture and/or using digitalfingerprinting. The best that can be done at this point is to digitallyfingerprint the object and use the techniques presented here to preservethat confidence going forward. Some embodiments may preserve this levelof confidence of authenticity by tying the digital fingerprint to themetadata about the item. In some examples, the metadata may include acertification of authenticity by the expert (or information thereof).

FIG. 11 illustrates a process 1100 of preserving a level of confidenceof authenticity of a physical object.

Blocks 1102-1106 may be similar to operations of the process describedwith respect to FIG. 1A. In diamond 1108, a database system (such as thedatabase system of FIG. 4) may determine whether the record isinitialized at creation of the physical object (e.g., at manufacture inthe case of manufactured goods). If the record is not initialized atcreation of the physical object, then in block 1110 the database systemmay retain in the database record an indication of no originalprovenance. The indication of no original provenance may be a binarydatum (such as a flag) in some examples.

The database system in block 1112 may acquire and store in the databaserecord authenticity information available in association with digitalfingerprinting. The authenticity information may include an indicationof authenticity, and metadata about the authenticity, such asinformation about a person that determined authenticity, howauthenticity was determined, when authenticity was determined, whereauthenticity was determined, or the like, or combinations thereof. Insome examples, the authenticity information may include a portion (e.g.,the entire or some of) an electronic certificate of authenticity. Theauthenticity information may include security information, such as adigital signature on the electronic certificate of authenticity.

If the record is initialized at creation of the physical object, thedatabase system may not retain the indication (block 1115). The databasesystem may of course retain information of the digital fingerprinting(not shown).

FIG. 12 illustrates a process 1200 of preserving a level of confidenceof authenticity of a physical object on induction into a trackingsystem. Data about a physical object may be inducted into an electronictracking system in block 1202. A database system of the electronictracking system may determine whether the data is inducted at creationof the physical object in diamond 1204. If the induction is not atcreation of the physical object, then in block 1206 the database systemmay retain in a database record an indication of no original provenance,which may be similar to block 1110 (FIG. 11). In block 1208, thedatabase system may acquire and store in the database recordauthenticity information available in association with induction, whichmay be similar to block 1112 (FIG. 11). If induction is at creation ofthe physical object, then in block 1210 the database system may notretain the indication (block 1210, which may be similar to block 1115(FIG. 11)).

Example Embodiments

Example C-1 is a method, comprising: acquiring digital image data of animage of at least a portion of a physical object; analyzing the imagedata to form a digital fingerprint of the physical object for anauthentication of the physical object, wherein the digital fingerprintis responsive to a natural structure of the physical object;initializing a database record associated with the physical objectresponsive to forming the digital fingerprint; determining whether thedigital fingerprint corresponds to original provenance for the physicalobject; in response to determining that the digital fingerprint does notcorrespond to original provenance for the physical object, acquire andstore in the database record first data including authenticityinformation available in association with formation of the digitalfingerprint; in response to determining that the digital fingerprintdoes correspond to original provenance for the physical object,retaining second data in the database record.

Example C-2 may include the subject matter of example C-1 (and/or anyother example herein), wherein the authenticity information includes anindication of a validation of authenticity and metadata for thevalidation.

Example C-3 may include the subject matter of example C-2 (and/or anyother example herein), wherein the metadata includes information about atime of ascertainment of said validation.

Example C-4 may include the subject matter of example C-2 (and/or anyother example herein), wherein the metadata includes information about alocation of the physical object at a time of the validation.

Example C-5 may include the subject matter of example C-4 (and/or anyother example herein), wherein the information about the locationincludes GPS (global positioning system) coordinates.

Example C-6 may include the subject matter of example C-2 (and/or anyother example herein), wherein the metadata includes an indicator for atype of the validation, wherein the type is selected from machine-basedand expert based types.

Example C-7 may include the subject matter of example C-2 (and/or anyother example herein), wherein the metadata includes a certificate ofauthenticity.

Example C-8 may include the subject matter of example C-7 (and/or anyother example herein), wherein the metadata includes a digital signatureon the certificate of authenticity.

Example C-9 may include the subject matter of example C-2 (and/or anyother example herein), wherein the metadata includes a source of avalidation service corresponding to the validation.

Example C-10 may include the subject matter of example C-2 (and/or anyother example herein), wherein the metadata specifies at least one ofmachine-based analysis, laboratory analysis, or microscopy.

Example C-11 may include the subject matter of example C-1 (and/or anyother example herein), wherein the digital fingerprint does not relyupon or require any labels, tags, integrated materials, uniqueidentification characters, codes or other items that were added to thephysical object specifically for the purpose of identification.

Example C-12 may include the subject matter of example C-2 (and/or anyother example herein), wherein the metadata includes a value taken froma label, tag, integrated material, unique identification character, codeor other item present on the physical object in association withacquisition of the digital image data.

Example C-13 may include the subject matter of example C-12 (and/or anyother example herein), wherein an edge of the label, tag, integratedmaterial, unique identification character, code, or other item appearsin the image and is represented by the digital image data.

Example C-14 may include the subject matter of example C-1 (and/or anyother example herein), wherein an authentication process artifact from alabel, tag, integrated material, unique identification character, code,or other item appears in the image and is represented by the digitalimage data.

Example C-15 may include the subject matter of example C-14 (and/or anyother example herein), wherein the authentication process artifactcomprises at least one of a inconsistency in a patina, wear markings,light bleach markings, or corrosion.

Example C-16 may include the subject matter of example C-1 (and/or anyother example herein), wherein an anti-counterfeiting artifactreferenced by an authentication certificate included in metadata of thefirst data appears in the image and is represented by the digital imagedata.

Example C-17 may include the subject matter of example C-16 (and/or anyother example herein), wherein the anti-counterfeiting artifactcomprises at least one of a inconsistency in a patina, wear markings,light bleach markings, or corrosion.

Example C-18 is a method, comprising: acquiring first informationcorresponding to a physical object; identifying first authenticationdata based on the first information; initialize a database recordassociated with the physical object responsive to identifying the firstauthentication data; determining whether the first authentication datacorresponds to original provenance for the physical object; in responseto determining that the first authentication data does not correspond tooriginal provenance for the physical object, acquire and store in thedatabase record first record data including authenticity informationavailable in association with identification of the first authenticationdata; in response to determining that the first authentication data doescorrespond to original provenance for the physical object, retainingsecond record data in the database record.

Example C-19 may include the subject matter of example C-18 (and/or anyother example herein), wherein the authenticity information includes anindication of a validation of authenticity and metadata for thevalidation.

Example C-20 may include the subject matter of example C-19 (and/or anyother example herein), wherein the metadata includes information about atime of ascertainment of said validation.

Example C-21 may include the subject matter of example C-19 (and/or anyother example herein), wherein the metadata includes information about alocation of the physical object at a time of the validation.

Example C-22 may include the subject matter of example C-21 (and/or anyother example herein), wherein the information about the locationincludes GPS (global positioning system) coordinates.

Example C-23 may include the subject matter of example C-19 (and/or anyother example herein), wherein the metadata includes an indicator for atype of the validation, wherein the type is selected from machine-basedand expert based types.

Example C-24 may include the subject matter of example C-19 (and/or anyother example herein), wherein the metadata includes a certificate ofauthenticity.

Example C-25 may include the subject matter of example C-24 (and/or anyother example herein), wherein the metadata includes a digital signatureon the certificate of authenticity.

Example C-26 may include the subject matter of example C-19 (and/or anyother example herein), wherein the metadata includes a source of avalidation service corresponding to the validation.

Example C-27 may include the subject matter of example C-19 (and/or anyother example herein), wherein the metadata specifies at least one ofmachine-based analysis, laboratory analysis, or microscopy.

Example C-28 may include the subject matter of example C-18 (and/or anyother example herein), wherein the first authentication data does notrely upon or require any labels, tags, integrated materials, uniqueidentification characters, codes or other items that were added to thephysical object specifically for the purpose of identification.

Example C-29 may include the subject matter of example C-19 (and/or anyother example herein), wherein at least one of the metadata, the firstinformation, or the first authentication data includes a value takenfrom a label, tag, integrated material, unique identification character,code or other item present on the physical object in association withacquisition of the digital image data.

Example C-30 is database system, wherein one or more processors of thedatabase system are configured to perform the steps of any one ofexamples C-1 through C-29 (and/or any other example herein).

Example C-31 may include the subject matter of example C-30 (and/or anyother example herein), wherein a first one of the one or more processorsoperates in a mobile device of the database system, and wherein thefirst processor is configured to perform the steps of C-18, or any ofthe other example of examples C-1 through C-29 (and/or any other exampleherein).

Example C-32 is one or more memory storing instructions to be executedby one or more processors, wherein the instructions when executedperform operations corresponding to the steps of any of the examples C-1through C-29 (and/or any other example herein).

Example C-33 may include the subject matter of example C-32 (and/or anyother example herein), wherein at least a portion of the one or moreprocessors comprises a processor of a mobile device.

Preserving Authentication Under Item Change

Some embodiments may include episodically updating a most currentdigital fingerprint to enable an object to be tracked even as it changessignificantly. Some embodiments may include the creation of a history ofthe item including the modified digital fingerprints (which may beindicative of wear, oxidative damage, or other changes) and metadatasuch as the accelerative forces, temperature extremes the item has beensubject to, humidity extremes the item has been subject to, pressure(e.g., atmospheric) the item has been subject to, or the like, orcombinations thereof, so that a more complete history of the item can beassembled. Some embodiments include the generation and use of historymetadata.

Many items do not change significantly from the time they are created totheir ultimate consumption. This is particularly true where supplychains and product lifetimes are short. Under such circumstances, thechanges that occur in the item while in the supply chain are essentiallythose of wear and tear from shipping, marking, and other modificationsthat occur as a side effect of the item's life cycle. However, for someitems these changes may be sufficiently severe as to reduce the matchbetween the authenticating digital fingerprint and the original one.

All items to some extent but particularly items that have long lifetimescan undergo significant changes that have nothing to do with wear andtear or transport. Parts can rust, wine bottle labels can corrode,credit card security regions can be abraded, etc. These changes arepresumed to take place gradually (on a scale slower than over whichre-authentication may be used).

Some items may change sufficiently from manufacturing to authenticationfor a digital fingerprint match to degrade significantly. Even thoughsome digital fingerprinting may be quite resistant to changes thatprevent matches of portions of the item, nevertheless cumulative changeseventually modify the item sufficiently that the digital fingerprintmatch may become uncertain or fail altogether. For long-lived itemscorrosion and other such changes similarly gradually alter the digitalfingerprints that could be extracted from the item.

Some embodiments may repeatedly acquire digital fingerprints of an item,store them in a database in concert with all the metadata gathered atmanufacture and whenever the item is authenticated (or otherwise madeavailable for data collection) and also in concert with previouslyextracted digital fingerprints, and through this continual updatingprocess accomplishing the goal of tracking the item with high confidenceeven in the presence of significant change. It is to be stressed thatwhether or not such changes are gradual or sudden, it is presumed thatthe item is digitally re-fingerprinted at the point where confidence inmatch between the current and the original digital fingerprint is stillstrong but deteriorating. The record of the item thus contains either asequence of historical (and now-current) digital fingerprints, or simplythe most recent one.

Wine may be bottled in a bottling plant associated with the vintner.After each bottle is filled and a cork inserted, the top of the bottlemay be sealed with a foil capsule (hereinafter “the capsule”). A digitalfingerprinting system may image the top of the capsule and record thedigital fingerprints for later use in tracking and/or authenticating thebottle. Under normal circumstances, where little change is anticipatedin the item, that is sufficient. For items such as wine, however, thatmay have a lifetime measured in decades (if not longer—the oldest extantbottle of wine is from the fourth century), or for items that mayundergo continual, gradual wear and tear during transport, a process topreserve authentication under item change may be utilized.

Wine, particularly high-end wine, may change hands many times during itslifetime. The capsule (or label or whatever is being used forauthentication or tracking) may age or get worn with time, eventuallychanging significantly. The bottle of wine may be re-authenticatedmultiple times during its lifetime, for instance the bottle of wine maybe authenticated upon changing hands.

Besides re-authentication at transactions such as when the bottle ofwine changes hands, scheduled re-authentication (e.g., periodically, sayevery year) may be used to preserve authentication under item change.The capsule (or appropriate part) may be imaged and the digitalfingerprints extracted. The digital fingerprints may be compared to therecorded digital fingerprints in the database and the bottle of wine maybe authenticated. At this point a newly-extracted digital fingerprintsmay be added to the database for any reason such as 1) they are addedwhenever the item changes hands as a matter of practice, 2) they areadded on some work cycle, e.g. every year, 3) the match with theexisting digital fingerprints is significantly less good than inprevious authentications or when tested at creation, 4) because theauthenticator observes degradation, or 5) for any other reason.

The newly added digital fingerprint may be included in the databaserecord for that bottle of wine, along with when it was taken and othermetadata as desired. Provided the deterioration in digital fingerprintmatch is not too severe, this system may preserve the provenance of theitem even if it changes sufficiently so that without this system, itcould no longer be authenticated.

FIG. 13 illustrates a process 1300 of preserving authentication underitem change.

A database system (such as the database system of FIG. 4) may performblock 1302, which may be similar to the process of FIG. 1A in someexamples. The database system may perform blocks 1304, 1306, 1308, 1310,and 1312, which may be similar to operations of process 200 of FIG. 2 insome examples.

In block 1316, based on an amount of difference between the digitalfingerprints, the database system may update the database to output anew indication of a new match to the physical object for any new samplesthat are not matchable to the first digital fingerprint within the firstpredetermined similarity threshold provided the new samples arematchable to the second digital fingerprint with a second predeterminedthreshold. The database system may store an authentication result,metadata, etc. in block 1318.

FIG. 14 illustrates a process 1400 of classifying item change.

In block 1402, the database system may scan a physical object andcapture data at intervals over time (e.g., as the physical object movesalong a supply chain, as the physical item changes hands, at periodicintervals, or the like, or combinations thereof). In block 1404, thedatabase system may store the captured image data.

In block 1405, the database system may generate a sequence offingerprints based on at least some of the captured data. For instance,the process of FIG. 1A may be performed on the physical object at someor all of the times of data capture.

In block 1406, the database system may recognize incremental changes toa natural structure of a physical object attributable to at least one ofwear from use over time, additional markings to the physical object overtime, corrosion over time, or wear from transport. The incrementalchanges may be indicated by the sequence of digital fingerprints, andthe database system may perform the recognition based on the sequence ofdigital fingerprints.

In block 1408, the database system may analyze the metadata to classifyat least one of the incremental changes. In block 1410, the databasesystem may store classification information in the database.

Example Embodiments

Example D-1 is a method, comprising: providing a database system tostore a first digital fingerprint based on first digital image data ofan image of at least a portion of a physical object, wherein the firstdigital fingerprint is responsive to a natural structure of the physicalobject; wherein the database system is configured to output anindication of a match to the physical object for any samples that arematchable to the first digital fingerprint within a first predeterminedsimilarity threshold; acquiring second digital image data of an image ofat least a portion of a target physical object; extracting features fromthe second image data to form a second digital fingerprint; querying thedatabase system to seek a matching record based on the second digitalfingerprint; based on an amount of difference between the first andsecond digital fingerprints, update the database system to output a newindication of a new match to the physical object for any new samplesthat are not matchable to the first digital fingerprint within saidfirst predetermined similarity threshold provided the new samples arematchable to the second digital fingerprint within a secondpredetermined similarity threshold.

Example D-2 may include the subject matter of example D-1 (and/or anyother example herein), wherein said updating further comprises: addingthe second digital fingerprint to a database record for the physicalobject responsive to matching the second digital fingerprint to thefirst digital fingerprint.

Example D-3 may include the subject matter of example D-2 (and/or anyother example herein), wherein the database system is arranged to, inresponse to identifying a third digital fingerprint, query the databasesystem to seek a matching record based on the third digital fingerprint;and wherein query the database system to seek the matching record basedon the third digital fingerprint comprises compare the third digitalfingerprint to at least one of the first or second digital fingerprints.

Example D-4 may include the subject matter of example D-3 (and/or anyother example herein), wherein compare the third digital fingerprint toat least one of the first or second digital fingerprints comprisescomparing the third digital fingerprint to the second digitalfingerprint before comparing the third digital fingerprint to the firstdigital fingerprint or conditionally comparing the third digitalfingerprint to the first digital fingerprint based on the comparison ofthe second and third digital fingerprints.

Example D-5 may include the subject matter of example D-3 (and/or anyother example herein), wherein compare the third digital fingerprint toat least one of the first or second digital fingerprints comprisescompare the third digital fingerprint to selected digital fingerprintsof all digital fingerprints of the database record.

Example D-6 may include the subject matter of example D-1 (and/or anyother example herein), wherein said updating comprises: adding thesecond digital fingerprint to a database record for the physical objectresponsive to matching the second digital fingerprint to the firstdigital fingerprint, wherein responsive to the adding the databaserecord comprises a sequence of digital fingerprints, wherein a mostrecent digital fingerprint of the stored sequence comprises the seconddigital fingerprint and a least recent digital fingerprint of the storedsequence comprises the first digital fingerprint or a less recentdigital fingerprint; and performing a selection from the set based on anamount of difference between the first and second digital fingerprints;and retaining in the database system control information based on theselection, wherein said control information is to constrain next one ormore comparisons of a third digital fingerprint of a next query to asubset of the digital fingerprints of the sequence or is to constrain atemporal order for performing a plurality of next comparisons of thethird digital fingerprint to digital fingerprints of the sequence.

Example D-7 may include the subject matter of example D-1 (and/or anyother example herein), wherein the second predetermined similaritythreshold is different than the first predetermined similaritythreshold.

Example D-8 may include the subject matter of example D-1 (and/or anyother example herein), further comprising: identifying a third digitalfingerprint, the third digital fingerprint similar to the second digitalfingerprint within the second predetermined similarity threshold, thethird digital fingerprint not similar to the first digital fingerprintwithin the first predetermined similarity threshold; identifying asequence of digital fingerprints associated with the physical object,the sequence including at least the first, second, and third digitalfingerprints; based on the sequence of digital fingerprints, recognizingincremental changes to the natural structure of the physical objectattributable to at least one of wear from use over time, addition ofmarkings to the physical object over time, corrosion over time, or wearfrom transport based on incremental changes; generating history data tocharacterize the changes responsive to said recognition; and causing thehistory data to be displayed on an electronic display.

Example D-9 may include the subject matter of example D-8 (and/or anyother example herein), further comprising: identifying a sequence ofmetadata, wherein each metadata of the sequence of metadata isassociated with a respective one of the digital fingerprints of thesequence of digital fingerprints; correlating at least one of thechanges of wear from use over time, addition of markings to the physicalobject over time, corrosion over time, or wear from transport based onincremental changes based on the sequence of metadata; and storing aresult of the correlation in the database system.

Example D-10 may include the subject matter of example D-9 (and/or anyother example herein), wherein the metadata specifies locations of scansto generate the digital fingerprints of the sequence of digitalfingerprints.

Example D-11 may include the subject matter of example D-9 (and/or anyother example herein), further comprising: correlating the at least oneof the changes to the natural structure to transport along a supplychain based on location data of the sequence of metadata; and displayinginformation about the correlation.

Example D-12 may include the subject matter of example D-9 (and/or anyother example herein), further comprising: correlating the at least oneof the changes to a time period based on time data of the sequence ofmetadata; wherein the correlation to at least one of wear from use overtime, addition of markings to the physical object over time, corrosionover time, or wear from transport based on incremental changes is basedon the correlation to the time period.

Example D-13 may include the subject matter of example D-8 (and/or anyother example herein), wherein the corrosion comprises oxidative damage.

Example D-14 may include the subject matter of example D-8 (and/or anyother example herein), wherein the wear over time comprises abrasion ofa security region of the physical object.

Example D-15 may include the subject matter of example D-14 (and/or anyother example herein), wherein the physical object comprises a creditcard.

Example D-16 may include the subject matter of example D-8 (and/or anyother example herein), wherein said recognition comprises detecting theaddition of material to the physical object to mark the physical objectbased on comparing different digital fingerprints of the sequence ofdigital fingerprints.

Example D-17 may include the subject matter of example D-8 (and/or anyother example herein), wherein said recognition comprises detecting theremoval of material from the physical object to mark the physical objectbased on comparing different digital fingerprints of the sequence ofdigital fingerprints.

Example D-18 may include the subject matter of example D-1 (and/or anyother example herein), further comprising storing the second digitalfingerprint in the database record.

Example D-19 may include the subject matter of example D-18 (and/or anyother example herein), further comprising retaining the first digitalfingerprint responsive to updating the database system to provide astored sequence of digital fingerprints associated with the digitalobject.

Example D-20 may include the subject matter of example D-1 (and/or anyother example herein), wherein the physical object comprises a bottle ofwine.

Example D-21 may include the subject matter of example D-20 (and/or anyother example herein), wherein the portion of the physical objectcomprises a foil capsule of the bottle of wine.

Example D-22 may include the subject matter of example D-1 (and/or anyother example herein), wherein updating the database record of thedatabase system based on the result of the determination furthercomprising replacing the first digital fingerprint with the seconddigital fingerprint in the database record.

Example D-23 may include the subject matter of example D-8 (and/or anyother example herein), further comprising: identifying a sequence ofmetadata; wherein each metadata of the sequence of metadata isassociated with a respective one of the digital fingerprints of thesequence of digital fingerprints, each one of the digital fingerprintsof the sequence and associated metadata formed and collected,respectively, at a different location of a plurality of locations alonga physical path of travel of the physical object; correlating the atleast one of the changes to the natural structure to a subset of thephysical path based on location data of the sequence of metadata; anddisplaying information about the correlation.

Example D-24 may include the subject matter of example D-23 (and/or anyother example herein), wherein the physical path of travel coincideswith a supply chain.

Example D-25 is a database system, wherein one or more processors of thedatabase system are configured to perform the steps of any one ofexamples D-1 through D-24 (and/or any other example herein).

Example D-26 may include the subject matter of example D-25 (and/or anyother example herein), wherein a first one of the one or more processorsoperates in a mobile device of the database system, and wherein thefirst processor is configured to perform the steps of D-1, or any of theother example of examples D-1 through D-24 (and/or any other exampleherein).

Example D-27 is one or more memory storing instructions to be executedby one or more processors, wherein the instructions when executedperform operations corresponding to the steps of any of the examples D-1through D-24 (and/or any other example herein).

Example D-28 may include the subject matter of example D-27 (and/or anyother example herein), wherein at least a portion of the one or moreprocessors comprises a processor of a mobile device.

Hardware and Associated Software

Most of the equipment discussed above comprises hardware and associatedsoftware. For example, the typical electronic device is likely toinclude one or more processors and software executable on thoseprocessors to carry out the operations described. We use the termsoftware herein in its commonly understood sense to refer to programs orroutines (subroutines, objects, plug-ins, etc.), as well as data, usableby a machine or processor. As is well known, computer programs generallycomprise instructions that are stored in machine-readable orcomputer-readable storage media. Some embodiments of the presentinvention may include executable programs or instructions that arestored in machine-readable or computer-readable storage media, such as adigital memory. We do not imply that a “computer” in the conventionalsense is required in any particular embodiment. For example, variousprocessors, embedded or otherwise, may be used in equipment such as thecomponents described herein.

Memory for storing software again is well known. In some embodiments,memory associated with a given processor may be stored in the samephysical device as the processor (“on-board” memory); for example, RAMor FLASH memory disposed within an integrated circuit microprocessor orthe like. In other examples, the memory comprises an independent device,such as an external disk drive, storage array, or portable FLASH keyfob. In such cases, the memory becomes “associated” with the digitalprocessor when the two are operatively coupled together, or incommunication with each other, for example by an I/O port, networkconnection, etc. such that the processor can read a file stored on thememory. Associated memory may be “read only” by design (ROM) or byvirtue of permission settings, or not. Other examples include but arenot limited to WORM, EPROM, EEPROM, FLASH, etc. Those technologies oftenare implemented in solid state semiconductor devices. Other memories maycomprise moving parts, such as a conventional rotating disk drive. Allsuch memories are “machine readable” or “computer-readable” and may beused to store executable instructions for implementing the functionsdescribed herein.

A “software product” refers to a memory device in which a series ofexecutable instructions are stored in a machine-readable form so that asuitable machine or processor, with appropriate access to the softwareproduct, can execute the instructions to carry out a process implementedby the instructions. Software products are sometimes used to distributesoftware. Any type of machine-readable memory, including withoutlimitation those summarized above, may be used to make a softwareproduct. That said, it is also known that software can be distributedvia electronic transmission (“download”), in which case there typicallywill be a corresponding software product at the transmitting end of thetransmission, or the receiving end, or both.

Having described and illustrated the principles of the invention in apreferred embodiment thereof, it should be apparent that the inventionmay be modified in arrangement and detail without departing from suchprinciples. We claim all modifications and variations coming within thespirit and scope of the following claims.

The invention claimed is:
 1. A method, comprising: providing a databasesystem having a processor; storing in the database system a firstdigital fingerprint based on first digital image data of a first digitalimage of at least a portion of a physical object, wherein the firstdigital fingerprint is responsive to a natural structure of the physicalobject and includes at least a first feature vector, wherein the firstfeature vector includes a series of numbers responsive to a region ofinterest of the first digital image data; acquiring second digital imagedata of a second digital image of at least a portion of a targetphysical object wherein the second digital image was captured at asecond time subsequent to a first time when the first digital image wascaptured; extracting, via a processor, features from the second digitalimage data, said features associated with a region of interest of thesecond digital image data; forming, via the processor, a second digitalfingerprint that is responsive to a natural structure of the targetphysical object and includes at least a second feature vector, whereinthe second feature vector includes a series of numbers responsive to theextracted features; querying the database system processor to seek amatching record based on the second digital fingerprint; in the databasesystem processor, responsive to the querying, determining an amount ofdifference between the first and second digital fingerprints, thedetermination of the amount of difference including comparing the firstand second feature vectors; in the database system processor, in a firstcase that the amount of difference between the first and second digitalfingerprints is within a first predetermined similarity threshold,returning an indication of a match to the first digital fingerprint; andin the database system processor, in a second case that the amount ofdifference between the first and second digital fingerprints exceeds thefirst predetermined similarity threshold and is within a secondpredetermined similarity threshold, generating an indication of a newmatch to the physical object, and storing the second digital fingerprintin the database system associated with the first digital fingerprint asan additional identifier of the physical object, and including in thedatabase system an indication of the second time that the second digitalimage data was captured.
 2. The method of claim 1, further comprising:in a third case that the amount of difference between the first andsecond digital fingerprints exceeds both the first predeterminedsimilarity threshold and the second predetermined similarity threshold,returning an indication of non-match.
 3. The method of claim 2, furthercomprising: acquiring a third digital fingerprint; in the databasesystem processor, comparing the third digital fingerprint to the firstand second digital fingerprints; based on the comparing steps, selectingone of the first and second digital fingerprints as a best match to thethird digital fingerprint; and in a case that the best match has anamount of difference that is between the first predetermined similaritythreshold and the second predetermined similarity threshold, adding thethird digital fingerprint to the database system as an additionalidentifier of the physical object.
 4. The method of claim 3, wherein thefirst, second and third digital fingerprints each include respectivefeature vectors and comparing the third digital fingerprint to at leastone of the first and second digital fingerprints includes comparing therespective feature vectors as one metric of similarity or closeness ofmatch.
 5. The method of claim 3, wherein comparing the third digitalfingerprint to at least one of the first or second digital fingerprintsincludes comparing the third digital fingerprint to selected digitalfingerprints of all digital fingerprints of the database system.
 6. Themethod of claim 1, further comprising: adding the second digitalfingerprint to a database record for the physical object responsive tomatching the second digital fingerprint to the first digitalfingerprint, so that the database record stores a sequence of digitalfingerprints, wherein a most recent digital fingerprint of the storedsequence of digital fingerprints includes the second digital fingerprintand a least recent digital fingerprint of the stored sequence of digitalfingerprints includes the first digital fingerprint or a less recentdigital fingerprint; performing a selection from the stored sequence ofdigital fingerprints based on the amount of difference between the firstand second digital fingerprints; and retaining in the database systemcontrol information based on the selection, wherein said controlinformation is to constrain one or more comparisons of another digitalfingerprint of a next query to a subset of the digital fingerprints ofthe sequence of digital fingerprints or is to constrain a temporal orderfor performing a plurality of next comparisons of the another digitalfingerprint to digital fingerprints of the sequence of digitalfingerprints.
 7. The method of claim 1, wherein the first and secondfeature vectors are multi-dimensional vectors and said comparing thefirst and second feature vectors includes calculating a distance betweenthe first and second feature vectors.
 8. The method of claim 1, furthercomprising: identifying a third digital fingerprint, the third digitalfingerprint similar to the second digital fingerprint within the secondpredetermined similarity threshold, the third digital fingerprint notsimilar to the first digital fingerprint within the first predeterminedsimilarity threshold; querying the database system to identify asequence of digital fingerprints associated with the physical object,the sequence of digital fingerprints including at least the first,second, and third digital fingerprints; based on the identified sequenceof digital fingerprints, recognizing incremental changes to the naturalstructure of the physical object attributable to at least one of wearfrom use over time, addition of markings to the physical object overtime, corrosion over time, or wear from transport based on incrementalchanges; generating history data to characterize the incrementalchanges; and causing the history data to be displayed on an electronicdisplay.
 9. The method of claim 8, further comprising: identifying asequence of metadata, wherein each metadata of the sequence of metadatais associated with a respective one of the digital fingerprints of thesequence of digital fingerprints; correlating at least one of theincremental changes of wear from use over time, addition of markings tothe physical object over time, corrosion over time, or wear fromtransport based on incremental changes based on the sequence ofmetadata; and storing a result of the correlation in the databasesystem.
 10. The method of claim 9, wherein the metadata of the sequenceof metadata specifies locations of scans to generate the digitalfingerprints of the sequence of digital fingerprints.
 11. The method ofclaim 9, further comprising: correlating the at least one of theincremental changes to the natural structure to transport along a supplychain based on location data of the sequence of metadata; and displayinginformation about the correlating.
 12. The method of claim 9, furthercomprising: correlating the at least one of the incremental changes to atime period based on time data of the sequence of metadata; wherein thecorrelation to at least one of wear from use over time, addition ofmarkings to the physical object over time, corrosion over time, or wearfrom transport based on incremental changes is based on the correlationto the time period.
 13. The method of claim 8, wherein the corrosionover time comprises oxidative damage.
 14. The method of claim 8, whereinthe wear from use over time comprises abrasion of a security region ofthe physical object.
 15. The method of claim 14, wherein the physicalobject comprises a credit card.
 16. The method of claim 8, wherein saidrecognizing incremental changes includes detecting the addition ofmaterial to the physical object to mark the physical object based oncomparing different digital fingerprints of the sequence of digitalfingerprints.
 17. The method of claim 8, wherein said recognizingincremental changes includes detecting the removal of material from thephysical object to mark the physical object based on comparing differentdigital fingerprints of the sequence of digital fingerprints.
 18. Themethod of claim 1, further comprising: storing the second digitalfingerprint in the database record.
 19. The method of claim 18, furthercomprising: retaining the first digital fingerprint responsive toupdating the database system to provide a stored sequence of digitalfingerprints associated with the physical object.
 20. The method ofclaim 1, further comprising: updating a database record of the databasesystem based on a result of determining the amount of difference byreplacing the first digital fingerprint with the second digitalfingerprint in the database record.
 21. The method of claim 1, whereinthe region of interest of the first digital image data is a specificfeature in the first digital image and the region of interest of thesecond digital image date is a specific feature in the second digitalimage.